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ABSTRACT 


The inclusion of time-based queries in video indexing application is enables by 
the recognition of time and date stamps in CCTV video. In this paper, we 
propose the system for reconstructing the path of the object in surveillance 
cameras based on time and date optical character recognition system. Since 
there is no boundary in region for time and date, Discrete Cosine Transform 
(DCT) method is applied in order to locate the region area. After the region for 
time and date is located, it is segmented and then features for the symbols of 
the time and date are extracted. Back propagation neural network is used for 
recognition of the features and then stores the result in the database. By using 
the resulted database, the system reconstructs the path for the object based on 
time. The proposed system will be implemented in MATLAB. 
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Because the two main problems in video OCR are complex 
background and low resolution, it is widely accepted [1, 2]. 
The text to recognize is not a mere sequence of characters, 
but a valid date and time. Different format of time and date 
have to be considered. The format includes time and date in 
one or two lines; date above or below; numeric or alphabetic 
month representation; seconds with one decimal digit; order 
of day and month in the date; different separators, etc. 

This learning approach has been shown to guarantee high 
rate of convergence and properties of stability and 
robustness of the solution. The recognition process starts 
with the search and the extraction of the portion of the 
original image. The characters contained in the plate region 
are localized by a robust processing using a non-traditional 
Discrete Cosine Transform (DCT) and subsequently 
segmented by applying low pass filtering on the plate region 
image and convert the gray scale plate region image into by 
using histogram-based shareholding method. After labeling, 
object with small area, which might be noise, are moved. 
From this segmented image we need to recognize the actual 
characters and numbers. Back propagation neural network 
is used for character recognition. 

PROPOSED SYSTEM 


INTRODUCTION 

In the last few years, optical character recognition in digital video has received 
an increasing interest [1]. It is widely used in many applications such as security 
control, supermarkets, banks, traffic monitoring, and etc... Some approaches 
exist and have been described in literature. They are mainly based on pattern 
matching and normalized correlations with a large database of stored templates 
[3,4]. The aim of the system is to recognize time and date stamps for allowing 
time-based queries and for reconstructing the path of the object. 


3.1 Localization of the Time and Date Region 

In the localization part, the entire image is input, and then 
the time and date region is localized by using block-based 
DCT (Discrete Cosine Transform). DCT transforms the spatial 
domain image into the frequency domain one. So, we can 
localize the time and date region from the DCT image 
because the numbers or characters on the region usually 
have high frequency. We detect the time and date region by 
scanning a window on the DCT image. If the number of 
horizontal, vertical and diagonal edges in the window is 
above a threshold value, then we assume that position is the 
res' 



In this study, the proposed system is based on the 
localization of time and date region and camera number, 
segmentation of these characters and recognition of 
characters and store these characters in the database. And 
then the system reconstructs the path of the object based on 
time. 


Fig 1 Example Input Image 

3.2 Segmentation 

In the segmentation part, we segment each number or 
character from the plate area image. First we apply low-pass 
filtering in the plate area image 3x3 averaging operator in 
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order to remove noise, then we convert the grey scale plate 
area image into binary image by using histogram-based 
shareholding method. The image size 640x480 pixels is used 
for input. First of all, all program read image file and find 
some information from that need to handle image. While 
reading processing ends, image data will be row image. And 
it is necessary to get only brightness information. After that 
image was converted into YCbCr color system and adopt 
only Y image. Interval of the window's movement is 4 pixels 
in row and column in order to reduce processing time. When 
we get plate image, which sized 170 x 90 to reduce error 
(each 10 more pixels in 4 directions]. 

It is required to know more about exact location of letters in 
plate so that some pre-processing steps have to be 
implemented. The first processing is 'binarization'. We used 
general binarization. The threshold value is very important 
thing. Next the extracted characters are converted into black 
and white characters represented by using binarization. To 
get a more accurate binarization results, this process uses 
several local threshold values since illumination of the 
plates, in general, is not uniform. 

Next to extract the plate characters and then proceeds to 
their identification, a labeling process is applied to the 
binary image, which is a very useful technique to segment 
characters with some inclination and different sizes. The 
labeling process allows the segmentation of the plate 
characters. After labeling, objects with small area, which 
might be noise, are removed. 

3.3 Feature Extraction 

The obtained characters were separately saved as an image 
file in feature extraction. A feature extraction process plays 
an important role in content based image classification. Back 
propagation ANN classifies the numbers and the letters for 
increasing the success rate of the recognition phase. The 
character images should be extracted as features before 
classification. These extracted features provide us to obtain 
the most discriminating information of an image. This 
information can be presented as a feature vector. The 
comparison between characters can be made when a feature 
vector that includes global and local features of a character 
should be encoded. In this system, the feature vector of an 
image was encoded by using Average Absolute Deviation 
algorithm [5]. This algorithm is defined as: 

V = (1) 

Where N is the number of pixels in the image, m is the mean 
of the image and f (x, y] is the value at point (x, y). The 
feature vectors with the length of 49 bytes for numbers and 
48 bytes for letters are obtained. Back propagation ANN uses 
the entire feature vectors as an input for classification of the 
characters. 

3.3 Recognition of the Characters 

Once each individual digit and/or character is extracted, it 
must be identified in some way. This process is called Optical 
Character Recognition, and there are several different 
solutions to this problem. The output of a generic OCR would 
basically consist of taking the characters that produce the 
highest matching for every segment in string. An 
interpretation of the result as a valid time and date is 
required. The numbers and the letters were classified by 
using back propagation ANN for increasing the success rate 
of the recognition phase. 


3.3.1 Back Propagation Neural Network 

In this approach, back propagation neural network is used. 
To improve the network until it is able to perform the task 
for which it is being trained, its iterative, recursive and 
efficient method for calculating the weights updates are used 
as main features of back propagation. Fig 2 is shown for a 
general block diagram of back propagation neural network. 
Firstly, the training parameters are defined: 

> Initial Conditions 

> Number of iterations 

> Number of hidden layer 
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Multilayer Neural Network which is based upon back 
propagation algorithm for training is a Back Propagation 
Neural Network (BPNN]. This neural network is based upon 
extended gradient descent based Delta learning rule which is 
known as Back Propagation rule. We must adjust the weights 
of each unit in such a way that the error between the desired 
output and the actual output is reduced to train a neural 
network to perform some task. This process requires that 
the neural network computes the error derivative of the 
weights. In other words, it must calculate how the error 
changes as each weight is increased or decreased slightly. 
The back propagation algorithm is the most widely used 
method for determining the derivative of Weight. If all the 
units in the network are linear, the back propagation 
algorithm is easiest to understand. In backward direction 
from output to hidden layer in this network to input layer in 
order to train the network, error signal between desired 
output and actual output is being propagated. 

Input layer, hidden layer and output layer are included in 
artificial neural network. Initially, training data are placed 
into the input layer. This is transmitted to the hidden layer 
and the output layer. The process is known as forward pass. 
The appropriate weight between nodes and generate output 
value of the resulting sum are computed and adjusted by 
each node in the input layer, hidden layer and output layer. 
After that the actual output values will be compared with the 
target output values. In order to update the weight of each 
node again, the error between these outputs will be 
calculated and propagated back to hidden layer. The error 
calculated is shown as in equation (2]. 

E=y 2 [dk-yk] 2 [2] 

Where dk is the output for neural network. 

Until the error is acceptable, the network will iterate over 
many cycles. For any new input data, the trained network is 
ready to use after the training step is done. The input for 
testing is placed into the input layer and results based on its 
knowledge from trained network will be generated by the 
feed forward network. The algorithm calculates each 
derivative of error by first calculating the rate at which the 
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error changes as the activity level of a unit is changed. For 
output units, the difference between the actual and the 
desired output is the error. We first identify all the weights 
between that hidden unit and the output units to which it is 
connected to compute the error for a hidden unit in the layer 
just before the output layer. Then, we multiply the error of 
those output units by those weights and add the products. 
The error for the chosen hidden unit is equal to this sum. 
Moving from layer to layer in a direction opposite to the way 
activities propagate through the network, we can compute 
the EAs for other layers after calculating all the errors in 
the hidden layer just before the output layer. It is straight 
forward to compute the derivative of error for each incoming 


connection of the unit once the error has been computed for 
a unit. In neural network, back propagation algorithm is 
most generally used. The input for testing is given to input 
layer and it will provide results according to knowledge it 
gets from training. 

After the recognition phase, the result of date and time and 
camera number are stored in the database. Finally, the 
system arranges the camera information based on time and 
date for the object that passed the cameras. In table 1, the 
object passed through GA-2030, GF-3065, GF1020, GF-2451, 
GA-2076 and so on. The flow chart of the proposed system is 
shown in fig 3. 



Extracted Video Frame 

Camera Information 

Point and Place 

Point No 

Year 

Month 

Day 

Hours 

Min 

Sec 

1 

2019 

3 

15 

12 

26 

6 

GA-2030 

Gate Point 

2 

2019 

3 

15 

1 

30 

34 

GA-2076 

Gate Point 

3 

2019 

3 

15 

12 

45 

12 

GF-3065 

Front of Building 

4 

2019 

3 

15 

12 

56 

23 

GF-1020 

Mid of Building 

5 

2019 

3 

15 

1 

20 

5 

GF-2451 

End of Building 

... 












Fig 3 System Flow Diagram 


CONCLUSIONS 

In this paper, we proposed the system for reconstructing the 
path of the obj ect based on time and date OCR in surveillance 
camera. Time and date OCR in surveillance system provides 
useful information. The integration of the proposed system 
in a bigger application will offer the ability of performing 
time-based queries. 
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